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ABSTRACT . a ce . - ee . 
oe ee ; Osing. the American Institutes. for Research (AIR) 
Report (1978) and. evaluation: of the Head Start. Bilingual/Bicultural 


Curriculum Project. by Juarez Associates as contrasting examples of 


bilingual research, the authors suggest.a new direction for policy 


‘research in bilingual education oriented towards examining all . 


_.. factors affecting such policy formation within,the larger relational ~ 

~-nexus. To provide «a context for the discussion, the Head Start 
“Strategy for Spanish Speaking Children and the plans for the | 
evaluation of its curriculua development component. are described. An 
outline of the-purpose and the design of the evaluation project 

4 follows. - This involves, not. only the adainistration of tests to | 

. " easure. the iepact of program objectives, but the coliection of data. 

“:  o. d1lustrate the. natare and éxtent of program. implementation over. 
tine, which-can then be used to determine the feasibility of - 

_ ipplementing the curriculum sodels at other. sites; and collection of 
"qualitative observation :and change: data over tise for use as outcome 
datas The reproductive function this. evaluation is. likely to serve is 
. then exemplified by drawing. attention to how. evaluative.-research can 
be used to legitimate social action programs. Puture directions are ~ 


also discussed. (Author/ABF) 
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7 ‘CuFrently,, 17 major national evaluation ‘Studies of Head 
coe a ae a er 
- “start. services. are underway. These. evaluations 2 4 an 


ce - a 


fe . : egare désigned. to. impfove the quality and: 
Bee : delivery of local ‘Head Start services. jn the oa 4 
. Se Ge 3 areas of education, parent involvémunt,;. Social: os 

7 4 -SOrvices,. health (including. medical, ‘dental, 3 
‘ ae ‘mental-health and. nutrition), and. services. an \ a a 
Ys, ae for':children with special needs--handicapped - 
we te 8 Spanish-speaking, Indian. and: migrant.~ (Head : 
_ ve tart Newsletters 1978. Pp. 6) ae . . 


s 


be One, of these At major studies. Zs An Evaluation: of the ‘Head oe 
eran Curriculuin Development Project (Juarez 
- arid: Associates, 1978)... This as a: major evaluation, and. it should. 
: not bé- taken ally by pode with an interest in bilingual édu- 
ae .. cation. “Ruthough it. ‘has a different purpose and. audience, given 
* TS stope- and. design, its impact may very wel]. make the American 


Institutes® for ‘Research (AIRS evaluation. of Title WIT programs 


(1.978) pale in. fomparison. _AS such, it ‘deserves close attention. 


‘Aside From the Fact. that! the Juarez and. Associates evaluation 
Bs 


(1.978): seems fore ‘promising ‘and interesting than the AIR evalua- 


“3s 


s 


, “tion, at ‘may - also be worth considering whether we can expest_ the 


*this is a revision of a paper presented at a forum on Ethno- 


“perspectives in Bilingual. Education Research held at eastern 4 
Michigan iniverstty. upeenets Michigan, Apri‘ 1979. : a 
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early ehildhodd ‘bi1ingual: education policy: formation, Although 


“this question Seems -prewaturé at this point, there is sufficient 


- thet, the: Substantive Findings may not realty matter. As -we-see— 


“yotytdnstansing radical. shifts in. .admini strative. leadership and _ « ae 


education “researchers.” 


serve a Reproduce ie: function and that the Jufrez and Associates 7 f 


“egedee to shape a context for. the discussion, we provide a brief 


. ym . : . 
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“Juarez “and “Wesrciates evaluation to inake a: difference in. shaping 


ne 
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3%, ee See ee aa: i; a 


reason to Glain that the findings of ‘the juérae and Associates ri 


evaluation. will Vikety’ be used as intended. The. ivony;:here is yf 


_ the evidence Seems, too. compelling: to conclude Otherwise. ee o: 


goverment policy; the evaluation fin ndings, will probably be used” “ ees 
6. legitimate federal, efforts in the. field-of early. childhood | 
DIV inguat. education. lovexpect-otherwice 15 to ‘overlook the 

structural linkage that exists between educational research and 
the. institutions that ‘Sponsor: it.. 


° 


The ‘general purpose of this paper is to. nivesent this evalua- 


‘ 


tion, yn an. open forum to allow greater discussion Of its poten- ° ‘ 
tial ‘impact on bilingual education. Thé Juarez and Associates . a 
evaluation “(41978) offers us ‘the attractive opportunity: of spe- - 


cifically looking at the. sreproductive function. of evaluative re- 


séarchy.: a. topic that should be of greater interest to bilingual ae E 


~ 


me ee en rere me 


We. argue that evaluative research can nad many times does - on 


iP ae : 
evaluation (1978). can. and probably will serve that function a 


maEner wey. The argument will proceed: as follows: First, if os 


description of the Head: Start Strategy for Spanish» ~Speaking. 


Chiddyen: and discuss generally the plans for evaluating its cur- . 
h, ; - . a : - 4 


Sere ey ae ae 


* . t pets Sales tae 
aan wore ccure ommended geaio oa am igh nate ae a rn ee i 


__tion for. Children, Y ‘outh and Famili ies (ACYE)--initiated a major 


rigulug devélopmient component. Second, we will afew ar 


poses and: gegen of. the. duanez and. Associates evaluation, 


etneeies eae to serve Spar aainG attention to how evalua 
tive research can ‘be used to legitimate social action programs. - 
{The wWestinghouse/omto evaluation (Datta, 1976) will “serve as an 


- examnpte. sa Finally, ‘we wid, conclude by ae t carve < out’ some: 


* / 


“difection. tor this ‘Kivi oF {Work .. 


HEAD. START STRATEGY FOR SPANISH-SPEAKING CHILDREN 


” 


Tt i's, “estimated that. approximately 19. percent: of ‘the total 
chitd Ponutation currently served by Head Start is Spanish sur- 
named. This. is. a sub- population of children with known variation 
along the dimens.ions, of language use and place of oricin, coming: 
‘from families ‘that are Mexican, Puerto Rican, Cuban, or Latin 
iAmerican in origin. eunide sone.speak limited English, many others 


speak. onky Spanish. 
In 1975, the Office of chits Development--now the adhintstrax 


program . to. address the specific needs of these ¢hildren. This 
“comprehensive program was named the Head Start Strategy ‘for: sa 
- Span nish=Speaking Children. This. strategy consists of éarrying 
“out ‘substantial ‘work in four felatively undeveloped areas of 

bilingual early chitdnood- education: staff training, bilingual/ 


jbiculturat ‘resource networks,, résearch, and curriculum develop- 


ment (ontracting Corporation ‘of, America, 1877). 


ros ee ee ee ene sort enn aera me cenetenerten senate Pe a 


quae Caan Ps ~ oes —. 


The ‘currént. work in basic ‘research, for examples. covers 


- @ Sen ae a 


. such. aFeas. a5 as; ea rly: ehiTaneod bitiagualism, ‘heart murmurs among. 
Mexican- Amer ican children, and parental resistance i diagnostic | 
"and! remedial heart care. The work in the area of staff training, 
‘ ait he other pand. neues? to the development. of four bilingual/, 
leila biculturalstaff tnadncing_models_ following the competency-based 
framework set forth in the Chitd Developrient Associate (CDA) pro- 
 oraey There is also. concurrent development of ‘two additional , 
models specifically designed to. include procedures for’ training 
bilingual/bicul tural} CDA trainers. To remedy: the lack of bilinguals 
bicultural resource networks, “a prototype network was set up‘in 
Defiver, Colorado to provide Head Start agencies ih Region VIII 
with in- service training, biljingual/bicultural materials, and 
; technical.assistance in the implementation of pi1ingual/bieuT tural 
. , programs. This prototype network was désigned and established 

to later serve as a replicable modell ae other regions. The: net- 
\ work is snow functioning, and the fakicnwide peplieacian is in 


. * process. ° " # o. 
ae the work in the are éa of curriculu ii development 
a --fotused specifically on the design of bilingual /bicul tural curri- 
cina\for preschool ‘children. In 1976, ACYF contracted with‘ four 
institutions--Columbia Teachers College, the High/Scope *Educa- 
: tional: Research Foundation, Inter- Cultural Development Research 
_ Associates; and the University of California at. Santa Cruz--to 
take on the task of developing, folie early childhood bilinguat/ 
7 bicultural sivvicuiun wodel's: The contractors. were expected to 
: work closely: with-local Head Start centers’ in a four-year develop- ; 


a? 
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sisted of mode? devel opment and. preparation for. training Head 

\ 
” start staff and Rarents to carry out implementation of the adel, 
expected: to occur: during. the second year \(1977- -1978) at the 


participating Head Start centers in the communities where iode1 


2 


_deveiopment was occurring: “Naw York ,. New Yorks Detroit, Michigans. 


a 


; San ‘Antonio, ! Texas; and. Watsonville, california. The third’ year 


two adda tional cites, The. fourth, year (1979- 1980) was uevotes 
“ae: the provision af continuing. 4upervision and technical ‘assis- 
tance to: the lead. Start centers at the replication. sites. 

Thus far, sevelopndee of the: models shows them to be funda- 
“men tatty: similar ih principle but dissimilar in- educational ap- 
prodens ANT the models explicitly acknowledge the importance 

- of attending 46 a ee aiid. the language and cultural 


needs of Spanish speaking children as bases for program design, 
but éach inode] ca:l-1s- for a different programmatic approach to the 


fducation of bilingual, children. These differences lare to be 
i 


expected;, for, the pedagogical principle structuring ‘the efforts 


| 


behind Head Start’ s bilingual/bicul tural ine aie is 


the “no single best approach. ® The director of, the, Head Start 


Strategy. for ‘Spanish-Speaking Children describes the principle 
as follows: : eee 


There is_no single "best" bil ingual/biéul tural 
preschool ‘program. -The precise form such: a 

program takes: should. be.a function of the’ group 
it serves. The cultural and linguistic. dif- 
ferences among Chinese, Native American, 'Spanish, 
Filipino and other groups are wide, and jthere 


{ 


1 . > 
. — 
: \ 
t : : ‘ 
; 
Ob 


ment process. , “The. work plan for. the first year (1976- 1977) : cons Sa 


Y (1978: 1979), called. for model: replication in Head. Start. centers at . 


a a : = : 5 
are oo ate: differences ¢ even within ‘each group. A 
: ; : - preschool: program serving Chinese- American - 
children in San. Francisco, for example;—. mF 
‘may not be appropriate for a Chinese-American ‘ 
, group. in New York: “Furthermore, several. 
4 racial and Yethnic groups, with or without 
, a different Tanguagé “or: dialect; may be 
represented in a. particular community. g Often 
y the reality of a preschool program is that 
it is bilingual and: mul ticil tural.—the- 
specific style and content of each -program 
: _ must, be tailored. to. the, ‘needs of the Ccom= 
“ee Bie and “the: groups within it. (Arenas, 1978, a, 
ie ip. sh: . F 4 : mop. 


Of fhe four comiponents in the Head Start Strategy for ae. 
Spanish- =Speaking canon the cufrigulum development project 
is the ‘only: ‘one heing. evaluated; and plans for evaluating the 
‘bFoducts of this “no single best approach" principle were set in _ 
‘motion in 1977. -ACYF gontracted with Juarez and Associates (a 
consulting firm based in Los Angeles, California) ig eonduee 
an evaluation ‘of. the Head Start pil ingual/Bicul tural Curriculum 
Development Project." The--eva Luation was to unfold in four 
phases over a three-and-a- half year period. The first phase 
(September 1977- October 1978) was devoted to selection of the a 
:  Heagurenant ‘battery and the sites expected to particdpate “in the 
evaluation (replication. sites “of model developers). In addition, \ 


a o “© 


the evaluators ~were ‘supposed to collect initial pretest data from 

teachers. During the second hase (October 1978- September 1979), 

ene seve tue torS were ‘expected: to pig.ot test the measurement battery 

and ‘randomize subjects into experimental and control groups.. Col- Seas 
léction -and analyses of data were to bé performed during the third 


phase (197.9-1980).. Finally, the fourth phase (October 1, 1980 to i 


2 


‘March 31, 1987) was to be devoted to arreing a final report and. 


° 


é 
, he cae : ve 


shaping the knpwledge, distribution system used’ to inform program, 
_ decision- making: at the local Head Start level: a set of pamphtets 
‘that describe Sach model in detail and discuss, among other things, 


requirements for its taplenentatién, 


a 7 me 
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“GOALS: OF THE JUAREZ. AND ASSOCIATES EVALUATION : 


wy 


The ‘quiding:purpose of the Jua pee an nd Associates evaluation. 


(1978) was to siipply ACYF wisth- information on the effectivengss 

_ of. ‘four Berl). childhood ‘bilingual /bicultural models for openly 

epee bet. children. Once the information was. supplied, ACYF 

planned. to use it in deciding on, the feasibility of wider distri- 

“bution Ot aie models to other Head Start centers across the nation. 

This basic information need on the part of: ACYF required that the 

evaluation focus on the collection of, three _types of data. 

Firsit, the “evaluation must find out.whether it is feasible 

: to successfully implement the curriculum. models in more than one 

setting. ° To. addgess. ‘eu question, the evaluators conleceae 

information ‘bearing, on\ oe general factors related to program 


4 


iinplementation: the process required for installing the program, 


v 


and ane program. phoceduses required of the model for activating 
its. valuéd message Systems. L Among the models! procedural fea- 
‘tures on which the evaluators expect to focus are staff, community, 
resource, and student characteristics required by the model for 


2 


a - es a n 


1s 


a“ 


The concept of message systems is a Bernsteinian notion that is 
used to refer to curriculum, pedagogy, and evaluation. For a 
detailed discussion, see Bernstein, 1975, pp. 85-115. 


ts 
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it to function. aS designed. 
‘ peconds the- evaluation must find out whether the model ob- 
jectives are being met by measuring impact on children, teachens, 
and: parénts. The contractual sqresment. 4 expiiett as to how 
this kind of eH SuECREN must be carried out. ACYF eae newiieed 
, the evaluation ‘to employ an. experimental pre- and posttest design 
‘in measuring model impact. At each réplication site, a total of 
90 four-year-old children were to be randomly seeigued to treat- 
ment and%éontrol groups. Following this. random assignment, the 
children ‘would ‘be. pre= and posttestied on a selected. number of 
| - competency. measures: (1) Spanish language Comprehension, 
: : ¢2) Spanish. language production, (3)- English language -comprehan- 
sion,. (4) English language production, (5) concept development, 
and (6) socio-emotional development. , 
To measure model ‘impact on teaghers: pre/post ‘interviews 
__were! planned,—designed—to—tap-background characteristics a“ 
ttitudes toward, knowledge of, and competency in early child- 


hood ‘bilingual /bicultural education. The same procedure was 


used to measure model impact on parents., The plans included pre/ 


saomeecberneiinim ote ae a 


post administration of a questionnaire designed to tap “packground 


characteristics and attitudes toward, and knowledge of; general _ 


education, bilingual/bicultural education, and vocational careers” 
for their cnildren. 


Lastly, the evaluation must find out whether, Head Start staff, 
parents, and: other community members have received the curriculum 
models favorably.- This information need was met by gauging 


Staff and parental satisfaction with the model. Parents, for | 


* 


i 


"each model. ACYF wilt then use these pamphlets to inforti ‘local gs 


Head srart centers. of. the options available to them when deciding : 


“Follow Through" was not among the goals of the: evaluation. The 


example, were expected to comment on the degree tc which the 
curriculum ‘modes were compatible with their views, of how their 


children should be boas ca Teachers, on the other hand, would 7 ‘ 


be asked to report on the quality, iaeity. suitability, and oe 


. perceived effectiveness of, ‘the model . 


A goal of ail, this research activity is the production of . ts 


pamphlet summaries describing the evaluation findings bearing on 


° 
* 


on how. best to meet the a needs of Spanish- ~speaking children. 
The aanpliets would describe ‘each model, discuss ‘what each would, 


require for implementation, identify what. each would offer in the 


form of programmatic SErOngEDS » and provide data ‘on how effective 


‘each hoded was fotind to be with children, sieneness and teachers. 


It is not the intent of the. pamphiets to provide a comparative 


analysis, of four models, sincé such an analysis. of the’ models a = ’ 


model comparisons were being left to the consumers of the models. 4. 


" DESIGN OF THE JUAREZ AND ASSOC.ATES EVALUATION 


* 


‘The design of the Juarez and Associates evaluation (1978) 

reduces to what they cal? a “holistic or multi-method data col- * 
lection strategy." It has also ane ‘referred to as a "contextual 
evaluation study design." For evaluators, these terms aré 


interchangeable; and they all imply an a2xpansion of the experi- 


mental approach so as to include ratings, event sampling, -and. : 


ethnographic techniques. The. inclusion of these other approaches 


o + a Se 2 


- inte thé. evaluation design specified by the ‘s¥aluatiton contract, 


the: argument. BOERS: would altow for drawing relationships between 


“program: ‘Processes (input) and outéome measures (output). This 
‘argitient- stems ‘from four, key, issues related to ‘evaluative research. 

_ ANI. four issues relate to the need of having: élear. and exact . _ 
knowl edie, of: (1) program ‘treatment, ke) Program caneleapante” 
_and/ Settings, (3) Prograit contexts »- and - (4) control group activi- 


"ties. In isolating these four issues, Judrez antl Associates 
j 


(1978). draw on a growing: consensus, among. re sada de eae the ack 


Seman 


. of these data 7s Vikely to make research fin dings, (particularly 


psychometric test. results) uninterprétable. This is the kind of 


a 


. s reasoning, thatihas helped shape the design of the so- -called con- 


: 


textual evaluation study of Juarez and AsSociates. _-. 


There are Enpee Pay objectives to. this type of study design. 


“The First: is to. measure the nature and extent of program imple- - 


* mentation. over time in: arden ‘to demonstrate “the feasibility of 
e implementing, the curriculum models at varying sites. Specific 
i a ae te., “on setting ‘and material resources, individuats, schedule and 3 
spear ee organization; attitudes: nd actual program activities 


: were te pe “jncluded ° “in "analyses employing a° fideltty ‘Perspective 


“in. the inéa surement of model implementation. i 


The second objective of the. design is to. secure the kind of * 


observational data of use in interpreting test outcomes. etn this , 


* . ° 
* a) . . = ve . s 


4 


2 iehouahi. the intent ‘eens is not to provide. a critique - of. the- 
evaluation design, there is &8 compulsion, however, to note. that 
the fidelity perspective. as used. traditionally in measuring im- 
plementation is problematic in securing accurate assessments of 
institutional change. See for example, Berman and McLaughlin 


(1974). 
at ee 1 2 : 


f 


-_ quantitative results. 


varying kind of qualitative data ‘that Juarez and Associates (1978) 


‘tangetéd for’ “collection: a 


° 


* 


Observations of classroom deeivities and situa- 


tions, such es specific lessons, and. language 
use in-Various situations, will be taken into 
consideration in the interpretatton-of test. ~ 
data. “These observations will also include 
information. on the measurement process or how 
the children in. both the treatment and ‘the- 


control groups: were. tested, and on the experi-#* 
". mentaT arrangement, including information on. 


the control group experience and how well ran- 
domization worked. All of the above activi- 


tTes. may have a. strong influence on west 


. 


over. time. 


é 


ag ‘outcome , 


results. ‘(I<p. 10). 


The intent here is not to use qualitative data‘ to 


data. As Ju4érez and Associates view it (1978) 


case, the qualitative data becomes an explanatory adjunct to - 


«] 


‘the third saieetive is, to secure qualitative change data 
\ 


es eeniain test results but to use qualitative data itself 


? 


The. more interesting feature here. is the 


uge éf qualitative data as. outcome. data. isa conp ienentary and 


pact. The 


being able 


the very real and commonly acknowledged possibility of side effects 


equally vatid way of providing “information bearing on model im- 


strategy has the added and very attractive feature of 


e 


to tap benitewatc outcomes. This is an important 


pointy for it illustrates the sensitivity of the evaluation beware 


dae ‘to explicit as well as implicit features of program designs 


(Monaghan. 


Side effects is’ an item. routjnely omitted by evaluation designs. 
7 / 
In sum, among the ls the qualitative data collection strategy 


were specific classro 


7 ¥ 


~~ 


1976). "Although important, the -measurement of program 


/ 


/- . ne . Ud 
/ : ° x . 7 
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om behaviors and ‘control group activities. 
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_ To. operationalize this. “multi -method désign, the evaluators 
Pane the use of a “par tie tant researcher" (PR). ‘This PRWaS 
Seen as very instrumental to the Juarez and Associates (1978) - ai 

scheme. One reason for suggesting the presence of a PR on site ‘ 
was: the need- to eliminate the "outsider" image. of eveTUSEOnS: 
fmm 


Ina “sense, PR ‘inight very,we TT stand for public relations in ‘this 


™ 


the PR was expected to get a deeper 


—e 


case. By bedoming an insider; 
view’ of program evaluations. While onsite, ‘the PR was expected 

to supervise psychometric testing, conduct naturalistic observa= a 
-tiods, “administér interviews. to téachers, and update credient and 
‘control group information: through periodie phone calls to parents. + 
As. proposed, the use of the PR is an interesting and intriguing sO 
innovation. in evaluation studies that reflects the strong in- 
fluences that ethnography ig havifig on evaluative research gener- 


ally and. on. ‘the ‘Juarez evaluation in particular. 
oe * TO. summarizé, the Juarez “and Associates ° (1978) evaluation 
déston contains many features critical to successful evaluations-- 
features ‘iackins in other federally. sponsored evaluations of bi- 
lingual education. There have been four well known federally - 
¥ ‘sponsored studies: (1) A: Process Evaluation of the Bilingual 
Education Program,. Titlé VII, ESEA (Development Associates, 1973). 
(2). Bilingual Edueatton: An pune Need (Comptroller seneral of 
the. Uni ted- States, 1976)3_ (3), the Condition of Bilingual Paucetton ; 
in the Hation (United. States Commi$s toner ‘of Education, 1976); and 
(4) mug taaeon of the Impact of ESEA Title VII Spanish/ingltsh 
Bilingual Sauedeion Program (Danoff, nS ‘Differences in pur- 


pose, range, and focus make it difficult ee compare these evalua- 


tions and the Judrez,and ASsociates (1978) study. Nevertheless, 
a brief summary of one of the most widely cited of these studies 
(the \AIR Report, 1978.) will ‘help to place the Juérez _study in 7 a 
ae The. Juérez effort represents an impact study of biline : 
qual education igith meiny of the Features hoped FOr in well- 

designed evaluations. We will examine how it differs from the 

Titte VII evaluation and. then explore how these differences in- 
étéa ce eke opportunity for the current study to suave a true : 
reproductive function... : : yO 

_ THE AIR REPORT, | = \ ee 


’ 


The. American Institutes for Research (AIR) conducted a major 
evaluation of Title VIL aa Interim results were released 
in. April, 1977; and, the Executive Summary of. -the. final report 
was released by the Officg of Education in March, 1978. These 2 
reports. were soon widely cited | as evidence that bilingual educa~ 7 
tion was 10% working. When the next year's education bill was 
debated, eendees eres Ashbrook of Ohio proposed ai amendment to 

aol ish- bilingual education, saying -that "the program is actually 
preventtng children from learning English.” ” The evaluation findings 


: , were used to SUPPOPE some Tawmakers' views of public policy; but, 


-for the most. part, Congress fas been supportive of bilingual educa= 
4 } : 


2 


fion, increasing the budget: annually. 


In the fall and spring of 1975-1976, thousands of students in 


“grades 2-6 were tested at 38 different sites across the country 
(in a total of 150 schools). The Title VII projects included in 


the study were in their fourth or fifth year of funding and pre- 


a 


‘sunéé ‘to be mature bilingual programs. . The comparison. group was = + 
ogeletted by. personnel who. each nominated one non-Title VII class- “ - 
rom with: “students comparable. to each Title vir classroom, The . | : 
fateh. was, in terms of ethnicity, socio- -economic deatud, ani. grade 
“level! “AIR reported that the non-Title VII students were basically 
éiparable to the Title VII students and that standard Statistical _ 
" grotedures were. ised ‘to agiuet 1 for differences BEEnCEn “Groups. 
Standardized. achievement tests were used to measure English, ‘and 
Spanish ‘Orat Comprehension and: Reading, and Mathematics. ‘Ab stue 
dent questionnaire obtained student background: information and: - 
“attitudes toward- -s¢hool werated activities. A sub= sample. of 7 4 i - 
" €lassrooms ‘were jusctes and teachers and directors interviewed. ~ ‘ 
to obtain. information. on the educational experiences of the stu- : 


\ 


dents (Danoff, 1978). - 
‘Three different weenods of analysis were used to assess. Tithe - a 
_WEE's. impact. Overall, "...the Title VII Program did not ‘appear 7 - oe 
to be ‘having. a consistent_ significant impactor. student achieve- . 
-ment in- these two subjects {English Language Arts and Mathematics ]" 
(AIR Report, 1978, p..17). _ The comparisons with. non-Title VII 
children’ showed that Title vit Students were performing worse in~ Seg : 
English than’ non-Title VII students. ae Math, the two groups 
were performing at about the same level. An increase fn Spanish 
Reading. scores was found during the year nOF Title VII students, 
but these gains could not be compared with the non- -Title VII : 


students because ene number who: could complete the test was. me 


small. . - | ¢- [ 
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a The study also examined. the educational goals. of Title VIT > 


on 


programs. Ine interviews with., directors. of: Title. ‘ViE-prograns, em 


an Py 


. 86 percent réported that children: remained in the: bilingual 

@ ae epodaees seven after ‘they. were able to funétion, in English. _Ac- 
|" cording. ‘to ‘the. AIR Report, "[t]hese Findings reflect Title VII 
_ project activities. which’ “pun counter a0 the. “trarisi tion approach 
strongly implied. by the SEA. Title WIT Tegistation® (1978. Pp. 42). 

. | - One. of the major difficulties ‘of most. Jarge- nscale educational. 
—_— evaluations is focating “and maintaining an appropriate comparison ~ 
e : g¥oup. “We. have the word. of the “AIR researchers that "elassrooms” 

= | Swere matched in terms of sts, ethnicity, and grade Vével: Yet the 

: Students: in Title VIT programs. did worse. on tests of ‘English than: 
their matched comparisons: Did AIR matgh the groups on initial , 


CY 


i, language facility, or language dominance? The report doesn't say, . 
% i t 
Jbut ‘it, was. reported that the Tnereases in Spanish Reading, scores — 


5 
1 
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ar 


of the “Tike VIL students could not. be. compared with the non- -Title 


Vit student" § scores because too ‘few- of the latter could even 


2 3 
x 


take the Spanish reading. test. ' ae oo . eee 
Groups were Judged for camparabiti ty in 1975- 1976, not when. . , 
the students began their Fitle VIT program. The ‘study also grouped 
all Tithe VII programs together with no attempt to identify and 
separately analyze quality bilingual programs. 
pong ‘Perhaps one -of the major weaknesses of the study is an ambi~ 


guity in its purpose. The United Statés Office of Education 


sought to° ‘discover +e impece oF Title VII programs (primarily on 
the children) when a prior oo might have ,been, "What educa- ~ 4 


tional. To are Peay ‘implemented under the Title: VII legis- ae 


a e . . - 
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é —Vation?! See fe 

ae. ~ The, politics of “bilingual ‘eduéation in- _the- Uni tedtstates is 


Ss 


<_---paraiHtel-reterence to the: vintues-of the: traditvonaT *neTEing pot” i 


es 


up 


wet ees 


" ree aeareee 
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mot abretatea to the: politics. of “basic skil1s," and: this “climate 


affeéts reaction: to. the at ally In the current. debate regarding. 


a return to the "basics." one hears: reference to: the tradittonal. 


‘Aner ican Values inherent in. ‘the mechanics Of reading, writigs. and: 


“arithmetic. In: debates on. bid ingual, education, there is. thé 


process in Ameriéa. It ia net be the- achievement findings that. 2 


we 


cause “concern among educators pb but the Tanguagemainténance approach a? 


reflected, in the responses of the Title VII prtares directors that 


7 . ra 


concerns: opponents. of bilingual education. : o es" 
- < ae THE : REPRODUCTIVE. FUNCTION oF THe “JUAREZ “EVALUATION 
= There is no ‘guch. thing. as neutral research. undther of the ees 
; ponder of scientific interests ‘Habermas (1977). describes, or 
ide ody » there are. specific interests. underlying att: Fortis of 
nae —knowledge- productions ‘App le—(1977.) describes the problen_well: 
+ shat is ‘the. prevailing function of résearch? | hi “4 
; Isn't it inerely a process of data. production P ‘ 2a 
.. that tries to help us solve our day-to=day  _ . at 
problems? I am. afraid that our answer -here™” -.. ss 
a must’ be no, for an affirmative answer neg] écts = <3 
eae . 3 one. important yet too often forgotten social oer ae 
a. "Facts research creates. information for use : ne Me, 


8 they per tore. Too.. often ) they. perform a - rather 7 


ys : by somebody. It is. ‘sponsored. by and affili- 
ated with organizations. that - have a. stake; a 
_though ‘often not a. conscious” one, in the con- - 4 
_ tinued maintenance of their-and other more. a ceil 
powerful, institutions" basic modes of opera- . : 
' tion. Thus .-we must ask the ‘political- -ques- ‘ , er 
‘tion, “Why Ts: this - data produced?" One should ~ ; : ee 
not conceive. of ‘the products of educational ae a 
esearch apart from. the. institutional functions - ae Re 


= 
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-__ciintere sting. roles.cthey*act.. to. prevent com- * —-- tt 
ts mitted. educators from focusing on the contra- | . * 
= , - dictions within our institutions by forcing oo: 
. 7 attention: on official definitions: of pFoblems. ” i 4 
by 4 . ot (pe 118). -_ im a mG 
s . > 
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‘Evaluative research. is no Sxeepelon: to this. Indeed, nowhere 


; "is: the nebeeductive function of “research more as hee than in 


: evaluative studies, for: these essentially seek a measurement of 


. the: reproduct fve value of that. whi chy ‘they, study. Now that t social 


my 


erent eit a Lee testo =e ey = eet 
¥ 


_ Systems generally look to scientific naetonay)ty for: Tegitimacy -_ 7 a ; 


Sma 7 » 


(Habermas, 1970), the reproductive Sanetten “Of evaluative research 


“ne = naam 


cp oceania eS aN eee, Sree ee meen Sas ernie semaine — oo 


Po Me becomes increasingly important. The national evaluation of 
Project Head Start is ilqustrative of just’ how important a role 
‘evaluative. research can come to play in the social reproduction 


of a major: federal program: : 


“\ oy 
of eee The: controversy chat. by the Wes tinghouse/Ohio Evaluation 7 
at pretect Head-Star (Datta, 1976) is wert ‘known. The politieaT 


~ 


and. methodological issued that it raised. were many, and the ‘de- 


~ bate was heated and long. But now that- the controversial dust 


Shas gett3edand. a_nuaber_of significant events. in the history of 


+ 
- the evaluation have been reconstructed, we can see clearly t that. 


: he ties tinghouse/ Ohio, evaluation. Seems. to have served its function 3 
‘ i 
welli: it’ helped to preserve Project Head Start and deliver to its 
. . aoe ee oe t : : 


‘readership what it had desired all along. i co 


The most important feature of the Westinghouse/Ohio evaluation 


(Datta, 1976) was not “the: ‘set of findings that were generated ous ae 


7: but the Sone er ve jegitimacy. that it produééd for ‘the internal ey 


Jaane and: external expansion of Project . Head Start. That Head Start fe 
# *. . ; a 


* / 
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a ‘Summer programs were generaly ‘Tess. effective than full-year 


Programs and that the impact. on Head ‘start children tended not 


ese 


to persist into the ‘primary grades was of no surprise to the Head 
‘Start Projéct. As Datta (1976) points out: 


Evidence suggesting. that. summer: programs were 
‘less developmentally ‘effective than full-year 
programs and that the academic ‘gains of Head 
Start were not sustained had -been available ; ; 
almost since, the. beginning of Head. Start. ° « Se oo mee 
The greater benefits of full- -year Versus sum- 
= mer programs were evident also in the 1965- 
to-1969 data... smost advisors...cautioned that 
os brief programs | would have limited effects on 
een ce” Tariguages -cognition,~or ‘other ‘aspects of ‘in- 
i ‘ tellectual development. (pp. 144-146) 


The fact” that this knowledge wae available to Project” Head 


Start prior ‘to its evaluation helps to explain why the Head ‘Start 
‘Research “council, argued strongly against the design of the _ 

West inghduse/Ohio ‘Study. ° “Archoughy it was unable to ster the Se 
waste design: of the impact study, . the Head: Start Research. Council , 
at Teast managed to influence the study to sort, degree. | Measures eu 
of parental sa.isfaction and child affect and motivation became 


part. of. the measurement battery. To complement the study, ‘an _ = a 


i+ ree ten 


a ‘advisory board of axperts was: ajso established. _ ; i ; ‘ 
. The reaction of the Head Start Council indicates that the 

evaluation ‘of the project may have been feared. In ‘Lane ot 

_ the knowledge then available to Head Start, this. fear seemed Seae™ 
" justified. _ NEVEREIEL ERS ere: ‘remained a ‘need Po legitimate , 
expansion - ‘the Head Start project. Such legitimation came. . : 
from. findings and recommendations supposedly ‘based on scientific 

rational ity.. As. ‘sources of this- rationality, the roles played 


“ne 


by, the Westinghouse Learning Corporation ‘and Ohio University can- 
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‘not, ‘bé\ under=estimated. | A more accurate understanding Gr athe 


West inghouse/Ohio evaluation requires relational analysis: that 
the. report not -be examined. apart from its producers--two institu- 
tions interested and committed to preserving and expanding research. 
: That. the evaluation. served a reproductive function is suggested 


‘by the Office of Child Development leadership as it reconstructs. 


~ =—“—gha History Of the Westinghouse/Ohio evaluation: 


We used Westinghouse and other studies con- 

- . gtructivély to reaffirm the -Head: Start focus 
pec ep - ~ * on-the:-whole\ chi-1d-+-the: concerns: about the .-— 
: “4 . Summer: projects were reasserted and the short- 
; term innoculation notion -wes shown to be falla- 
: cious; the notion of a single magic year was 
also. shown to be limited; -and, the folly of 
selecting as criteria stable measures of devel- 
opment was emphasized... 


sos 7 . Westinghouse is just. the ‘only ‘book on- the shelf 

bee = uv —-- about--Head-S tart;-and--so-peopte point to it. 

: But. the issues. debated about Head Start and 

— = early childhood education: are issues we'd have 
been debating anyhow. _ 

Sa Westinghouse helped justify what we wantedi as — 

early as 1966: continuity, earlien involvement, 

o greater concern about parent involvement. Anda 

,- we used Westinghouse to legitimize these. (Natta, ee 

“1976, p. 151) i 


dh ee ale ee 


It is true that the Head Start budget stabilized after the 


"  " weport. It is aiso true that Elliot Richardson, Secretary of 
the Department of Health, Education, and Welfare under Richard 

oe Nixon, may have. received convincing pressure from parents to 

| preserve Head Start. But there are two factors we need to keep 

ig Inu First, the demonstration of Head Start parents in 

Washington buttressed one of the few positive findings of the 


West inghousé/Ohio evaluation -(Datta, 1976): parental satisfaction, 


_.addéd' as a measure to the study, at the insistencé of the Head 


= 


a 
~ a 7 


- Start Research Council. In a sense, the report may -have func- 


“pe tioned. to legitimate the, action of the parents as welt. Second, 


te 
‘ 


policies in Washington were beginning to shift as a. result of new : 


administrative leadership. The untimeliness of the report almost 


interfered with Vts reproductive function. The new leadership at 


. the Department of Health, Education, -and ‘Welfare found itself 7 

“holding the results. of a study it had neither sponsored nor Bean 
prepared ‘to use. The evaluation had losts ‘its political legi tie 

» --~ “Macy==a- casé of Scientific rationality without political sick ie 

The political legitimacy of the evaluation had to be reconstituted 


jn the form of both federal. and grass roots support. % 


In the! ‘case of the Juérez.and Associates evaluation (1978), 
there is. a key factor, strongly. favoring its reproductive function--_ 
. the. structural Linkage that exists ‘between the Administration for 
a Children, Youth and Families (ACYF) and the evaluation: On the 
one hand, ACYF. wants to help Head Start programs desiring to imple- 


ment Bilingue curricula to is able to select, the model that best 


— ‘suits their local needs and is compatible with their staff $: 

-~--——-teaching style" (Arenas, 1978=1975, -p.. 18):. This implies a very : pee 

: different use of information. than making. budget. decis ions about a ic 
national bilingual education program like Title VII. On the other 
hand, a consulting firm consented to address the. al prob- 
“Len that ACYF had defined. and to design 5 carry out - "an evalda- 
tion to.resolve it. The result was an saavucunai spectticaliy 

; salu to ‘alan findings. that local Head ear programs could 


% 
’  »but under what He ircamstanead ant needs" each worked: or failed to 


Work, ee 


ii yen 


parts. ‘This is the: uu of conélusion that tends, at least. politi- 


; j \ \ . 
<pOse and the ‘more “open” design of the Juarez and Associates evalua- 


were funded for a limited time period and limited in size (four 


“sites and only hundreds of thousands of dollars spread over several 


VII program. The Juarez and Associates findings can be used by a 


Although the AIR Report (1978). obtained some program descrip- 
tive data, this information was not sufficiently precise to permit 
an assessment of existing circumstances and under which. ones partic- 
ular effects would be found.. Thus, we are left at the end of the 
AIR Répurt*with basically one globai conclusion: Title VII children 


do no ‘better, and> ‘perhaps worse, than their non-Title VII eouncer= 


cally, to weaken the “reproductive power of an evaluation. When it 


an 


Nap RENE: the loss. in reproductive power then has to\be. “restored 


we 


through recommendations of some kind. Because of the different pure 
tion (1978), the reproductive function of that evaluation may not. 
need the doctoring usually found in evaluations in the form of "posi- 
tive" recommendations suggesting that the program ber saved: a 

The structure) linkage between ACYF and the Juérez and heeos 
ciates evaluation (1978)° makes the study more focused and more 


limited, focusing on a particular set of curriculum projects that 


& 


years). In no way does it approach che multi-million dollar Title 


single agéticy to shape its own strategy toward bilingual children 


and. families and does not have to serve as "the" evaluation of 


6 


"United States bilingual education." a 


o, *. 
4 


Furthermore, ACYF has a history of supporting program develop- 


ment-and research efforts, raising the probability that findings 


from the Juarez and Associatés study will be received positively 
, ; ae “4 


22. 
4 |e 
and acted on by the agency. In 1972, for example, ACYF (then 
the Office of Child Development) began Home Start, a home-based 
variant ‘of Head Start and simultaneously funded an evaluation . . 
designed jg, collect both process and outcome data for judging 
the’ ‘implementation and impact of the program. Information: from 
ve the evaluation was used. in modifying guidelines for the program? 
and. when Home Start ended and more than 300 Head Stang programs 
adopted e home-based option (usually to ‘Supplement their angotnd 
center- eused program) ; ACYF instituted six regional training 
centers to provide training and. technical assistance to Head Start 
‘programs’ making this change (Love, 1978). If the Juarez and . 
Associates evaluation (1978) provides useful information about 
the four bilingual curricGium projects, there is every indication ° 
that ACYF will attempt, to base future programmatic decisions on 
L that information. ” _ te 4 
It: seems to us that mandat2s to evaluate institutional pol- 
icies and practices presents a troublesome dilemma to policy- 
“makers, depectaliy those with an fnterest in preserving certain 
ae policies and practices considered important and worthwhile. _ The, * ~ at 
: Tpolicyaaker hee ‘to. demonstrate the worth of a given policy “or oo 
program to justify its preservation. How does the policymaker do ~ ° 
this without jeopardizing the policy or program? One option is 
for. the policymaker to realistically and clearly define what he/ 
she wants the evaluation to do and then support, a Wen thought- 
out ‘evaluation design capable of delivering the kind of Aa ia _ 


usable in. maintaining or improving “that portey or program. That 


is the posture reflected in. the auahez ‘and Associates “evaluation ~ 


“= 
° . . 
. 


(1978), and: we submit .that itis as reasonable as any other. 
"4 TOWARD A NEW DIRECTION. IN BILINGUAL/BICULTURAL 
EDUCATION POLICY RESEARCH _ : 


There is a dire need*in bilingual/bicultural education: policy 
research for ‘engaging in relational analysts. This perspective 
: -requires that all social activity be construed as structurally 
7 related to -otker forms of social activity and institutions. ‘Social 
actions are defined ir verms of their relations to other forms of 
social actions and underlying principles of social organization 
\ and control. Apple (1979) describes this research approach as 
wt follows: | | 


What I am asking for is what might best be 
vt - called "reiational analyses." It involves 
: seeing social activity--with education as a 
particular form of that activity--as tied to 
the larger arrangement of institutions which 
apportion resources, so that particular groups 
and classes have. historically been. helped 
while others have been less adequately treated. 
In essence, social actisn, cultural and educa-. 
tional events and artifacts (what Bourdieu 
would call cultural capital) are "defined" 
not by their obvious qualities that we can 
jmmediately see. Instead ‘of this rather posi- 
tivistic approach, things are: given meaning 
rationally, by their complex ties and con- 
nections to how a society is.organized and 
controlled. ,,The relations themselves are the 
defining characteristics. Thus, to understand, 
say, the notions of science and the individual, 
. as we employ them. in education especially, we 
need-to se them as primarily ideological: and.. 
» economic categories thatsare essential to .-: 
both the production of agents to fill existing 
economic roles and the reproduction of disposi- 
tions and meanings in these agents that will 
"cause" them to, accept these alienating voles - 
without too much-questioning. They become 
aspects of hegemony. (p. 10) a 2 


on 
? 


“The absence oF this view in ‘bilingual education research is 


a. bit ironic. AS reformers, bilingual educators have done their 

. share. ee secentuets ana eliminate inéquities in educational oppor= 
tunity: ‘As researchers, however, they have not given equal. atten- 
tion. to discovering the ae reasons for these inequities... 
The: tendency has been to reiy more on. developing languages of” 
incompati bilities ‘and. ‘bieognitivisn to justify institutional 

; amelioration rather than. restricturation: une is, with legiti- 
acs ‘ mating the. improvement, of practice as opposed to creating different 
. “structural aprangements. that are more responsive. The problen a 
“here is an these janguage ay ebens function more as sages. and 
tend to fail short of what is needed for understanding the struc- ” 
‘tural relations underlying sécial and cultural reproduction. 
“What is needed isa different metaphor or language system 
“that, atiows us to. describe the structural relations between insti- ~ 
tutiois and individuals so that we can see how individuals and — 
institutions. function for and against each other. The kind of 
language. system we have in mind here is central to the. sociology — 
of school knowledge being developed by Willis (1977), Eggleston 

y (1977) » Young, (1971), Bernstein (1975), B Bourdieu and Passeron 
(1977), ‘Green and Sharp (1975), and Apple (1979). cn 

q “In the work of, these theorists we find a metaphor virtually 

* absent trom the Titerature on bilingual education. This metaphor 
Tinks ‘educdtional praeeieds “including research, to other social 
-procésses and- institutions in the society. Education, for example; 


Je Seen as essentially a mechanism of cultural preservation and 


df stribution that hejps to create and recreate forms of conscious- 
. : . ‘ \ 


ESS, \ Education As thus. ‘the ‘means by which a society reproduces - 


a ‘consevous ‘awareness of seduce Ors Aceording to Apple (1979). 


the sot aacaecusteiee of hegemony best describes the ralationship 


. Ba 


oo er 5 4 | - 
poate The key here, though, is “that this reproduction Ts subtle, 
remeinyng largely ~unconscious on the part of educators, rather 
than as -the result “of ante Tite group ‘of managers who sit around 
tables. plotting ways to "do in" their workers. in the workplace 
and school. Educational activity and:interaction are so thoroughly 


tied the patterns, of the wider society that they are outside. 


cbetweenjthe assumptions and behaviors of educators. on the oné -hand- 


‘and. the | istruature ‘and. peEterns of the wider society on the: other. 


i¢ally, hegemony: implies that the fundamental, economic and 


Umar 


poditical divisfons. in a society © "¢  «& 
- .eeare held: together by tacit aaaeleaical . 
-aSsumptions,. cor-riiles, which are ‘not usually 
conscious..’.. These rules serve to organize 
: and legitimate the "activity of the nfany 
j individuals whose interaction makes up a 
. social order" “(Apples 1979, p. 86) ‘ ; . 
The ‘a's sumptions necessary for maintaining the existing social ” order: 
_89 thoroughly saturate the activities ‘of Sioee within the society 
“that ‘they actually. define whee is common sense; individuals be- “ 
have in a manner that reinforces the social order, not. because 
they are forced but because they cannot conceive of any ‘other way 
‘to. act. In the realm of education, Apple (1979) sees. hegemony 
“Manteastee in the deployment by educators of intel Léptuat tradi- 
tions. for both legitimating and transmitting existing practices.’ 
He calls. ehis: phenomenon. "ideological saturation. It can be 


illustrated with analyses of how ane true role ‘of conflict in 


' 


t ) 


- 
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-used to control .and ultimately victimize whole classes of children. 


“Again, Appte is careful to warn. against construing his argument 


“patterns. Consider the debate over the Title VII findings, for 
na . \ 


. the critical issue that has been raised: whether the federal ea 


ad 


f 
’ 


social.»change. is glossed over in sciencé curricula and by exploring 


the language of diagnosis and labeling used in elementary schools. 


Ce BP De ot a nee et ey ot tty 


In .both cases, Apple shows how these "selective traditions" are 


to imply conscious conspiracies among educators: 


This is not to imply, that all school people 
are racist (though“some-may in fact be) or 
that they are:.part of a conscious conspiracy 
to: "keep the Tower Classes in their place." 


‘ 


-In fact, many. of the arguments for "community" Re A 
and. about-curriculum put forth by ‘some. of jot  s 
the early educators, curriculum workers, and - ee 
-intel-lectuals...were based: on the best liberal a. 
intentions. of "helping: people." Rather the 4 


“argument being presented here is that "natu- 

__ rally". generated out of many..:educators.-..— Z 
common-sense: asSumptions and practices: about > 
teaching and Yearning, normal and abnormal 

‘behavior; important and. unimportant knowledge, 
and so forth are conditicns and forms of in- 
teraction. that have latent functions. And ,. : 
these latent functions include some things 
that many educators. are -not -usually aware of. 
(Apple, 197.9, pp. 64-65) - : : 


The metaphor of education as cultural reproduction can help 


Gx 


veer ee een ee emma 
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us better understand how evaluative research is constrained by” 


and contributes to the maintenance of existing sociocultural” 
instance. Throughout this debate, we have somehow overlooked 


government should play a mediating role in the reproduction of 
ethnic languages and cultures. We can't respond to that issue 

by. pointing to a faulty evaluation design. The issue is politi- - 
cally charged, and research’ design has very little to do with it:, . 


The issue relates more to a structural relation that federal 


‘- 


be + 
t 


support for bilingual education seems to be endangering--a poli-— 


2 4 
. 


tical concern that the Title VII evaluation helped to exacerbate 


4a showing show: bilingual. programs were being used for language 


* 


~ and: ‘cultural maintenance. : sie 


+ To alleviate. the concern, federal ‘pol jeymakers have started 


rr < e 2 


a search, for entry- -exit critéria applicablé to Title VIE students. 
This. ‘search is certainly a good indicator that the Title VII 


= evaluation. has served. its reproductive function rather weil. in 


~ 


certain areas. The: rind ings: have so ‘far led to tighter: monitoring 


-and. control that are, in a-sense, inprovenants and therefore 


FEN YROIEN of the ‘program. _-It_makes sense, therefore, to. pays 


‘less attention to questions of faulty design and more attention 


to the: structural reasons underlying. the Title VII evaluation; 
“wiién looked at closely. the problem with the AIR evaluation -has 
: more to. do with the purpose and type of questions raised than > 
“ with faulty. design. | | 
; To summarize, using the AIR Report (1978) and the Juarez and 
Associates (1978) evaluations as contrasting examples of bilingual 
“research; we have suggested..a different direction for policy 


“4 research in bilingual education. , We suggest an orientation that 
: . 
looks at all the factors: (program evaluations being one example) 
& 
“affecting bilingual education policy formation within the larger_. 


relational nexus of which they are.a parts If these factors are 
to: be: sccunatery interpreted, we cannot afford to settle for 


~— 


anything less. 
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